High-speed router backplane

ABSTRACT

A high-speed router backplane is disclosed. The disclosed construction and layout techniques enable the construction of a reliable, high-layer-count, and economical backplane for routers and the like that use signaling across the backplane at trace speeds of 2.5 Gbps or greater. Specific ranges of differential trace geometry characteristics, with significant single-ended coupling to adjacent ground planes, have been found to provide the parameters needed for such signaling. New trace routing and layering techniques also help in the realization of a backplane embodiment containing roughly 600 operable high-speed differential pairs, while also providing sufficient electromagnetic interference management to allow power distribution to occur within the same backplane.

FIELD OF THE INVENTION

This invention relates generally to backplanes, and more specifically tobackplane wiring systems for highly interconnected, high-speed modulardigital communications systems such as routers and switches.

BACKGROUND OF THE INVENTION

A backplane generally comprises a printed circuit board having a numberof card connection slots or bays. Each slot or bay comprises, e.g., oneor more modular signal connectors or card edge connectors, mounted onthe backplane. A removable circuit board or “card” can be plugged intothe connector(s) of each slot. Each removable circuit board containsdrivers and receivers necessary to communicate signals across thebackplane with corresponding drivers and receivers on other removablecircuit boards.

One or more layers of conductive traces are formed on and/or in thebackplane. The traces connect to individual signal connection points atthe various slots to form data lines and control lines.

Router backplanes present a challenging area of circuit board design(for convenience, routers and switches will be referred to hereincollectively as “routers”, as the technical distinctions between the twoare unimportant to the invention as described herein). By their verynature, configurable modular routers require a high degree ofinterconnectivity between their removable router cards. With anyappreciable number of cards, it becomes infeasible to build largeparallel point-to-point connection buses between each pairing of thecards. This limitation hinders further growth in large routerthroughput, as the next generation of large routers may well seethroughput requirements measured in terabits-per-second. As suchthroughput requirements may require several tens (or even hundreds) oflogical ports to exchange data simultaneously at twenty to one-hundredGigabit-per-second (Gbps) speeds, it can be appreciated that theconnectivity and throughput requirements placed on large routerbackplanes are extreme.

Many router manufacturers, believing that the limits of electricalcircuit boards have been reached in the area of large router backplanes,are now designing optical backplanes for their next-generation products.Optical backplanes avoid some of the most problematic characteristics ofelectrical backplanes, such as trace density, signal attenuation, signalreflection, radiated noise, crosstalk, and manufacturinglimitations—characteristics that become increasingly significant assingle-trace signaling speeds push into the multi-Gbps range. Opticalbackplanes, however, come with their own set of problems, chief amongthese being cost and complexity.

SUMMARY OF THE INVENTION

This disclosure describes an electrical router backplane that overcomesmany of the electrical and mechanical limitations of large prior artelectrical backplanes, and methods for its design and fabrication.Generally, this backplane comprises multiple high-speed signaling layersof differential signaling pairs, separated by ground layers. Preferably,power distribution layers and/or low-speed signaling layers are embeddednear the center of the backplane stack, between outer groups ofhigh-speed signaling layers. Various additional design features can becombined within this general architecture to produce a backplane thathas been tested for reliable communication at single trace pairdifferential-signaling speeds up to 10.7 Gbps, 200-ampere powerdistribution, and overall backplane throughput greater than 1.6Terabits/second.

In the present disclosure, a wide range of new backplane features andmanufacturing processes are disclosed, each of which contributes to theoverall success of the backplane design. Preferably, these aspects arecombined in a single backplane to provide an accumulation of thebenefits of each aspect.

BRIEF DESCRIPTION OF THE DRAWING

The invention may be best understood by reading the disclosure withreference to the drawing, wherein:

FIG. 1 contains a block diagram of a high-speed router;

FIG. 2 illustrates one possible path for traffic entering a router atone line card and exiting the router at another line card;

FIG. 3 shows the external layout for a router backplane circuit boardaccording to one embodiment of the invention;

FIG. 4 shows the same layout as FIG. 3, with superimposed internaldifferential pair trace routing for the connections between one linecard and one switching fabric card;

FIG. 5 shows several high-speed differential signal trace pairs passingthrough a card connector region of a router backplane;

FIG. 6 shows a high-speed signal trace pair on one signaling plane of arouter backplane connected to a pair of signal thru-holes, with a loopin one trace to equalize trace length;

FIG. 7 shows the panel mask for one power plane of a router backplane;

FIG. 7A contains a magnified section of the mask of FIG. 7, showing anisolation cutout used for isolating mechanical equipment power fromrouter card power;

FIG. 7B contains a magnified section of the mask of FIG. 7, showing aguard ring that surrounds the power plane;

FIG. 8 shows the complete material stack in cross-section for a routerbackplane according to a hybrid dielectric embodiment of the invention;

FIG. 9 shows the complete material stack in cross-section for a routerbackplane according to a singe-dielectric-material embodiment of theinvention;

FIG. 10 illustrates a cross-section through one section of a routerbackplane high-speed signal layer, illustrating a trace/groundplane/dielectric layer configuration according to an embodiment of theinvention;

FIG. 11 depicts an ideal eye diagram for a differential signal;

FIG. 12 depicts a fully open eye diagram for a differential signal;

FIG. 13 depicts a non-preferred eye diagram illustrative of what mightbe expected with a backplane without the tailored stub capacitance ofFIG. 15;

FIG. 14 plots an eye diagram typical of a router backplane operating at3.125 GHz with signal thru-holes as shown in FIG. 15;

FIGS. 15 a and 15 b each illustrate a signal thru-hole and a ground holein cross-section for a router backplane according to two embodiments ofthe invention;

FIGS. 16–19 illustrate various pad and clearance configurations usefulwith the signal thru-hole of FIGS. 15 a and 15 b;

FIG. 20 shows the panel mask for a high-speed signaling layer in arouter backplane;

FIG. 21 illustrates a partial cross-section of a router backplane,illustrating how shorter differential pairs connect at shallower signallayers and longer differential pairs connect at deeper signal layers;

FIGS. 22 a and 22 b show, respectively, exploded and assembled views forthe first lamination cycle of a two-lamination cycle process;

FIG. 23 contains a magnified portion of the mask of FIG. 20,illustrating thieving used in an embodiment;

FIG. 24 shows the panel mask for a digital ground plane in a routerbackplane;

FIG. 25 illustrates thieving for two adjacent low-speed signalinglayers; and

FIG. 26 illustrates a teardrop signal pad used on low-speed signalinglayers.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

1 Definitions

Several terms have been assigned particular meanings within the contextof this disclosure. As used herein, high speed signaling refers tosignaling on a differential signal pair at a data rate greater thanabout 2.5 Gbps. A high-speed signaling layer or high-speed differentialtrace plane contains high-speed differential signal trace pairs, but mayalso contain lower speed and/or single-ended traces. A core dielectriclayer is one that is cured and plated prior to assembly of a circuitboard. A b-stage dielectric layer is one that is cured during assemblyof cores into the circuit board. Differential signaling (or balancedsignaling) is a mode of signal transmission, using two conductors, inwhich each conductor carries a signal of equal magnitude, but oppositepolarity. Single-ended signaling (or unbalanced signaling) is a mode ofsignal transmission where one conductor carries a signal with respect toa common ground. The impedance of a differential trace is moredifferential than single-ended if the impedance between that trace andits differentially paired trace is less than the impedance between thattrace and ground.

2 Basic Router Configuration and Operation

By way of introduction, one type of router configuration that can takeadvantage of the backplanes disclosed herein will be described. FIG. 1shows a high-level block diagram for a router 20. Line cards 30, 40, 50,and 60 provide physical ports to the device. For instance, line cards 30and 40 can each provide up to 24 Gigabit Ethernet ports 22 into router20. Line card 50 provides two 10-Gigabit Ethernet ports 52, and linecard 60 provides an OC-192 POS (Packet-Over-Sonet) port 62. Althoughfour line cards are shown, many backplanes provide slots to accommodatemany more cards, e.g., up to fourteen line cards in the embodiment shownin FIG. 3. The user can configure device 20 to accommodate differenttraffic capacities, traffic models, and physical port mixes by theappropriate selection of numbers and types of line cards.

Switching fabric 70 switches each routed data packet from that packet'singress port/line card to that packet's egress port/line card. Switchingfabric 70 connects to each line card through two full duplex switchingfabric port connections (see, e.g., port connections 44, 46 to line card40). Switching fabric 70 can be reconfigured rapidly on anepoch-by-epoch basis (an epoch is a defined time slice). For instance,at one epoch, fabric 70 may be switching packets from ingress port 44 toegress port 54 and from ingress port 46 to egress port 66, and at thenext epoch, fabric 70 could be switching packets from ingress port 44 toegress port 64. At any given epoch, ingress ports and egress ports arepaired to utilize as many switching ports as possible without undulydelaying a particular set of packets.

In an embodiment using the backplane of FIG. 3, the switching fabricfunctionality is distributed among nine identical switch fabric cards,eight of which are ganged to actively switch packet data in parallel(the ninth provides redundancy). In this configuration, a full-duplexswitching fabric “port” actually comprises 18 differential pairsconnected to a line card—one transmit pair from the line card to eachswitch fabric card, and one receive pair from each switch fabric card tothe line card.

Route processing module (RPM) 80 resides on an RPM card. RPM 80 hasseveral duties. RPM 80 is responsible for overall system operation,i.e., recognizing and booting new line cards, identifying faulty linecards, packet route discovery, and sharing routing table informationwith the line cards. RPM 80 also provides a user interface (not shown)to allow a system operator to configure the system and view systemparameters. For each of these functions, RPM 80 generally communicateswith the line cards over control bus 90. As compared to the switchingfabric ports, the control bus can be a relatively low-speed channel.

Another duty of RPM 80 is scheduling switching fabric 70. In a preferredimplementation, RPM 80 reconfigures switching fabric 70 every epoch. RPM80 uses scheduling bus 92 to communicate to switching fabric 70—as wellas to line cards 30, 40, 50, 60—the switching fabric configuration forthe upcoming epochs. RPM 80 attempts to schedule as many fabric ports aspossible during each epoch, and to ensure that data is handled promptlyand fairly. As compared to the switching fabric ports, the schedulingbus can be a relatively low-speed channel.

RPM 80 also maintains its own switching fabric port connection 82,allowing it to receive and transmit packets external to the router usingany of the line card physical ports. In the backplane design of FIG. 3,provision is also made for a second RPM card connected to router 20 toprovide failover capability.

FIG. 2 shows an exemplary data path taken by part of a packet as ittraverses router 20. FIG. 2 depicts three cards that would be insertedin a typical system—an ingress line card 30, an egress line card 50, anda switch fabric card 70 a. Note that a fully functional system wouldalso contain at least seven additional switch fabric cards and at leastone functioning RPM card, but these have been omitted from FIG. 2 forclarity.

Cards 30, 50, and 70 a are shown connected to a backplane 100 usingboard connectors and sockets, of which the numbered connectors 35, 55,75 and numbered sockets 37, 57, 77 are typical. The board connectors arepress-fit onto their respective cards, and the matching sockets arepress-fit onto the backplane. A card then can be connected to thebackplane by mating the connectors with the sockets at a desired cardslot. Other connectors (such as connector 39) located at each slotperform functions such as supplying power to a card.

The number of integrated circuits and division of circuitry functions ona card can be varied in many ways, as this is not critical to thepresent invention. In FIG. 2, line card circuitry is illustrated in onepossible configuration: an ingress circuit (31 and 51) for processingpackets received at the line card, an egress circuit (32 and 52) forprocessing packets to be transmitted by the line card, and a serdes(serializer/deserializers 33 and 53) for passing packets between theingress/egress circuits and the switch fabric cards. Switch fabric cardcircuitry is illustrated in one possible configuration also: a switch 71in communication with a serdes 73 to pass packet data between switch 71and the line cards.

One possible data path through router 20 is shown in FIG. 2. An incomingpacket PacketIn is received at a port on line card 30. Ingress circuit31 processes the packet, determines that the appropriate router egressport is on line card 50, and queues the packet in a queue correspondingto line card 50. At an appropriate epoch, one data path of switch 71 isconfigured (along with the corresponding switches on the other switchfabric cards, not shown) to switch data from line card 30 to line card50. During that epoch, serdes 33 receives the exemplary packet's datafrom the queue, serializes it, and transmits a portion of that data toeach switch fabric card. Serdes 33 transmits the portion of that databound for switching fabric card 70 a over a physical path comprisingconnector 35, socket 37, differential pair 34 a in backplane 100, socket77, and connector 75. Serdes 73 receives that data, de-serializes it,and passes it to switch 71. Switch 71 switches the data to anappropriate channel for line card 50, and then passes the data back toserdes 73. Serdes 73 reserializes and transmits the data over a physicalpath comprising connector 75, socket 77, differential pair 56 a inbackplane 100, socket 55, and connector 57. Serdes 53 combines theserial data received from the switch fabric cards and passes thede-serialized data to egress circuit 52. Egress circuit 52 performsadditional packet processing, and queues the packet for transmission outthe appropriate egress port as PacketOut.

3 Backplane Lateral Layout

The description of the backplane design is divided into two sections.This first section describes aspects of the embodiments of the inventionas they relate to the lateral layout of the backplane. A second sectionwill describe aspects of the embodiments of the invention as they relateto the backplane cross-section design.

3.1 General Arrangement

FIG. 3 shows a detailed backplane-plating layout for a router 20 andbackplane 100 as described in FIGS. 1 and 2. A top panel region ofbackplane 100 has connector regions (“slots”) for sixteen cards. Theoutboard seven slots on each end are each configured to accept a linecard (slots LC0 to LC6 and LC7 to LC13). The middlemost two slots areeach configured to accept a route-processing module (slots RPM0 andRPM1). Each slot has three upper connector regions (e.g., regions JL4U0,JL4U1, and JL4U2 for slot LC4) used to distribute power and groundsignals to a card. Below these, each line card slot has three high-speedconnector regions (e.g., regions JLC4A, JLC4B, and JLC4C for slot LC4).The RPM slots serve more card connections than the line card slots, andtherefore use a larger high-speed connector region. In one embodiment,the high-speed connector regions are laid out to accept HS3 press-fitsockets, available from Tyco Electronics Corporation (formerly AMPIncorporated).

A bottom panel region of backplane 100 contains connector regions orslots for nine cards. Each of these slots in configured to accept aswitch fabric card (slots SF0 to SF8). Each slot has two lower connectorregions (e.g., regions JSF8U0 and JSF8U1 for slot LC8) used todistribute power and ground signals to a switch fabric card. Abovethese, each switch fabric card slot has three high-speed connectorregions (e.g., regions JSF8A, JSF8B, and JSF8C for slot SF8).

The bottom panel region also contains connector regions for connectingpower and ground to the backplane. Two 48-volt power distribution layersare embedded in backplane 100, an “A” power distribution layer and a “B”power distribution layer. At the lower left of backplane 100, two largemulti-thru-hole regions 48VA and 48VA RTN allow for connection of “A”power supply and return leads to one power supply, and a third largeregion CGND allows for connection of a common ground. Similarconnections for a “B” power distribution layer to a second power supplyexist at the lower right of backplane 100.

3.2 Signal Distribution

One advantage of the layout of FIG. 3 is that it allows for efficientrouting of the high-speed signaling connections between the variouscards. The RPM card slots are centrally located, as they require thehighest number of connections. The switch fabric cards are alsocentrally located, but below the line cards and RPM cards, providingrouting room for the connections between the switching fabric card rankand the line/RPM card rank.

As will be described below, the preferred backplane embodiments utilizespecific combinations of high-speed signaling layers, low-speedsignaling layers, and power distribution layers to provide theconnections necessary for router functionality. The high-speed connectorregions of backplane 100 interconnect using the high-speed signalinglayers. Although connections that operate at relatively low speeds—suchas the router's control bus, scheduling bus, and clock distributiontraces—an also utilize the high-speed layers, separate low speed layersare preferably provided for them. The power distribution layers are usedto distribute power from the router's power supplies to the router'scards.

3.2.1 Differential Pair Configuration

High-speed signaling across the backplane preferably utilizesdifferential trace pairs. One aspect of the present invention thereforeinvolves the routing layout of differential trace pairs within thehigh-speed signaling layers. Achieving a workable routing layout usingpre-existing techniques would be, at best, difficult, due to severalother attributes of the preferred embodiments. For instance, thepreferred pitches for differential pairs are eight mil traces onsixteen-mil spacing and seven mil traces on seventeen mil spacing—muchwider than a conventional differential pair (e.g., eight mil spacing foreight mil traces) might use. This preferred pitch decreases achievabledifferential pair routing density on a given signaling layer. Also, inorder to limit maximum trace length and pack a large number of cardsinto a standard rack-mounted chassis space, routing space between cards(and their respective connector regions) has been limited—when combinedwith the preferred differential pair pitch, the resulting configurationallows only about five differential pairs to be routed side-by-sidebetween any two adjacent cards on a given signaling layer. Further, toensure manufacturability, the number of high-speed signaling layers isalso limited, e.g., to ten in this embodiment.

3.2.2 Typical Routing Paths

Within the constraints identified above, FIG. 4 illustrates high-speeddifferential pair routing for one exemplary line card slot to switchfabric card slot. Each line card slot (and RPM slot) connects in similarfashion to each switch fabric card slot for purposes of high-speedsignaling. As shown in FIG. 4, line card slot LC3 connects to switchfabric card slot SF7 via four differential pairs 102, 104, 106, 108. Twoof these differential pairs are used for high-speed signaling from LC3to SF7; the remaining two are used for high-speed signaling in theopposite direction.

In some embodiments, some (or all) differential pairs connecting switchfabric card slot SF7 to the other line card (and RPM) slots reside on acommon high-speed signaling layer with differential pairs 102, 104, 106,108. The non-illustrated pairs route in similar fashion to theillustrated pairs, with line card slots nearer the center of thebackplane connecting to SF7 nearer its top, and line card slots nearerthe ends of the backplane connecting to SF7 nearer its bottom. Left-sideline card slots generally connect to thru-holes on the left of slot SF7,and right-side line card slots connect to thru-holes on the right ofslot SF7. Generally, similar routing exists on the other high-speedsignaling layers as well (each layer generally serving one switch fabriccard slot), with one exception that will be described shortly.

3.2.3 Paths Through Connector Regions

Because in this type of embodiment a large number of differential pairs(more than 60 in some cases) route to one switch fabric card slot oneach high-speed signaling layer, at least some pairs must pass throughthe connector regions for other switch fabric card slots if a tight cardspacing and short switch fabric card height are to be maintained. InFIG. 4, pairs 102 and 104 pass through two connector regions each (SF5and SF6), pair 106 passes through three connector regions (SF4, SF5, andSF6), and pair 108 passes through five connector regions (SF3, SF4, SF5,and SF6). In a worst case, a differential pair may have to pass througheight switch fabric connector regions.

The connector regions are densely populated with alternating rows ofsignal and ground pins, again, in order to minimize space requirements.One aspect of the invention involves a particular way of routingdifferential pairs through the connector regions that largely avoidscrosstalk and signal attenuation. Exemplary differential pair routingsof this type are illustrated in the scale drawing of FIG. 5.

FIG. 5 illustrates in top view a segment 110 of a high-speed signalinglayer, showing typical scale spacing for the thru-holes in a connectorregion. This segment contains rows of six signaling pin thru-holes(e.g., holes 112 a–f in one row) alternating with rows of three groundpin thru-holes (e.g., holes 114 a–c in one row). The configuration istypical of a thru-hole pattern used with an AMP HS3 connector.

Each thru-hole is plated, with each signaling thru-hole used in thebackplane potentially carrying a high-speed signal. Thus the potentialfor signal interference exists each place that a differential pair isrouted past a signaling thru-hole. The preferred embodiments minimizethis potential interference by routing differential pairs throughconnector regions in an alignment that intersects a row of ground pinthru-holes (see, e.g., differential pair 116 a, 116 b). This places thedifferential pairs as far as possible from the neighboring signaling pinthru-holes, and at the same time largely maintains the desirableimpedance characteristics of each trace pair as it traverses theconnector region.

The traces of the differential pairs already route with approximately asix- to eight-mil spacing, measured vertically in the material stack,from adjacent ground planes. Consequently, very little nettrace-to-ground impedance effect results from passing such a tracehorizontally past a ground pin thru-hole (connected to those same groundplanes) by roughly a ten-mil spacing. Further, the use of traces thatare only slightly more differential than single-ended allows pairs oftraces to split and couple with a thru-hole without greatly affectingtransmission.

The preferred embodiments use a routing layout that splits adifferential pair as it approaches a ground-pin thru-hole, allowing onetrace to pass on one side of the hole and the other trace to pass on theother side of the hole. Once past the thru-hole, the two traces rejoinin the differential configuration on the opposite side of the hole. Thisapproach advantageously allows the differential pair to retain adifferential configuration along much of its path through a viaconnection region, while avoiding interference to a large degree withsignals present in signaling thru-holes in that via connection region.

In the preferred configuration (illustrated by differential pair 116 a,116 b), the centerline of the differential pair is aligned with thecenterline of the row of ground pin thru-holes (114 a–c). As traces 116a and 116 b approach ground pin thru-hole 114 a from the left, thetraces turn and separate at approximately a 90-degree angle (the tracesrouted respectively at plus and minus 45 degrees from their originaldirection of travel) until separated by more than the clearance requiredfor the thru-hole. The traces then turn back and pass the thru-holeparallel to each other, and rejoin again at approximately a 90-degreeangle until reaching the original differential configuration. Traces 116a and 116 b route in substantially the same manner around ground pinthru-holes 114 b and 114 c before exiting the card connector region.

Although the routing illustrated for traces 116 a, 116 b is preferred,other routings are possible. For instance, differential pair 124 a, 124b approach ground pin thru-hole row 126 a–c slightly off axis from theleft, allowing trace 124 a to pass just above thru-hole 126 a withoutturning. Trace 124 b turns downward to pass just below thru-hole 126 a,and then straightens out. After trace 124 a passes thru-hole 126 a, italso turns downward to rejoin trace 124 b in a differentialconfiguration. On approaching ground pin thru-hole 126 b, this patternis reversed as the differential pair jogs back upwards, and so on. Anadvantage of this routing is that it requires only half the trace turnsrequired by the routing of traces 116 a, 116 b. Disadvantages are thatthe traces are placed somewhat nearer some adjacent rows of signalingthru-holes, and the traces depart from their differential configurationfor longer segments.

FIG. 5 illustrates other useful differential pair constructs. Forinstance, differential pair 118 a, 118 b is routed through region 110 totwo signal pin thru-holes 120 a, 120 b. This differential pair remainsaligned with a row of ground pin thru-holes (122 a–c) until reaching theproximity of signal pin thru-holes 120 a, 120 b. The differential pairthen angles towards signal pin thru-holes 120 a, 120 b such that thedifferential configuration is maintained as long as possible.

3.2.4 Differential Pair Path Matching

Note that as described and shown in FIG. 5, differential trace 118 a isslightly longer than differential trace 118 b. In some situations, itmay be possible to reverse this configuration on the other end of thetrace pair, such that trace length is equalized. Removing path mismatchpresent at one end of a differential pair by an offsetting mismatch atthe other end of the pair is not, however, always possible orpreferable. Accordingly, FIG. 6 illustrates a trace terminationconfiguration for use in such situations. A looped jog 130 is placednear the source end of trace 118 b, thus approximately equalizing thelength of nominally shorter trace 118 b with the length of nominallylonger trace 118 a.

Note that as shown in FIG. 6, looped jog 130 more than compensates forthe extra length in trace 118 a. This extra length in trace 118 b alsocompensates for a path length difference in the AMP HS3 connector forthe pins that connect to traces 118 a and 118 b. Simply crossing thetraces could compensate for the connector path length difference, suchthat at the exit end of the traces trace 118 b connected to theconnector pin with the longer path length. In the disclosed embodiments,this is non-preferred. Otherwise, the rising signal edge on one traceand the corresponding falling signal edge on the other trace will bemisaligned over the entire backplane path, causing signal distortion anddiminishing the common-mode noise rejection capability of thedifferential pair. Note that it is extremely difficult to completelyeliminate misalignment between the rising and falling signal edges on adifferential pair, but the described jog technique greatly diminishesthe problem.

3.3 Power Distribution

The preferred embodiments utilize a novel power distribution schemeemploying four relatively thick conductive planes near the center of thebackplane for power distribution to the line and switch fabric cards.These planes provide a relatively noise-free and economic powerdistribution scheme for a router, as compared to more conventional powerdistribution approaches such as bus bars or separate power distributioncircuit boards. The present embodiments are believed to be the firstbackplanes capable of distributing 100 amperes or more of current toattached components (in the preferred embodiment, two distinct powerdistribution planes are each capable of distributing 200 amperes ofpower).

3.3.1 Power Blocks/Location

FIG. 3 shows the power entry/exit points for a preferred backplaneembodiment. The large plated regions 48VA, 48VA RTN, 48VB, and 48VB RTNprovide connection points for redundant A and B power supplies. Fromthese comer locations, power is fanned out to thru-holes for the switchfabric power connectors (e.g., JSF8U0 and JSF8U1) arranged along thebottom of the backplane and thru-holes for the line and RPM card powerconnectors (e.g., JLAU0, JL4U1, and JL4U2) arranged along the top of thebackplane. This arrangement is preferred, in part, because it leavesmore trace routing room near the center of the backplane for creatingshorter high-speed traces.

3.3.2 Via-Free Paths

FIG. 7 illustrates a panel mask for the 48VA power distribution plane,with dark areas representing areas where copper will be etched awayduring patterning. In FIG. 7 the panel mask has been turned ninetydegrees clockwise with respect to FIG. 3. It can be appreciated fromFIG. 7 that power distribution is enhanced by the existence of a largecentral substantially via-free path (populated in this embodiment by arelatively few holes for board alignment pins). This region, lyingbetween the leftmost line card/RPM high-speed connectors and therightmost switch fabric high-speed connectors in FIG. 7, provides a widepath capable of distributing several hundred amperes of current.

The areas between the high-speed connectors for adjacent cards are alsosubstantially via-free. This allows power distributed through thecentral via-free path to channel freely between adjacent rows ofhigh-speed connectors in order to reach power blocks near the top andbottom of the backplane.

One aspect of the via-free path concept is an adherence to a routingdesign that avoids layer-swapping vias. In other words, every signalinjected at a thru-hole to a given signal plane of the board isextracted from a second thru-hole to that same signal plane, with nointermediate via(s), connected to two signal planes, that swaps thesignal to a different plane. A layer-swapping approach is often taken inthe prior art to solve routing problems, but is specifically avoided inthe preferred embodiments of the present invention. This not onlyimproves power distribution, but also avoids the creation of extraneousreflections due to intermediate vias in high-speed signal paths.

3.4 Noise Suppression

As mentioned previously, one advantage of the embedded powerdistribution layers of the preferred embodiments is enhanced noisesuppression as compared to conventional methods of power distribution.Some aspects of this noise suppression relate to the layer ordering ofthe backplane, and will be discussed in Section 4. Other aspects relateto the horizontal plan of the power distribution planes, in particularthe use of isolation cutouts and the use of a copper guard ring.

3.4.1 Isolation Cutouts for Fan Power Distribution

In the preferred embodiment, the backplane distributes power not only tothe switch fabric, RPM, and line cards, but also to power connectors fora complement of fan trays that provide convection air-cooling for therouter. Consequently, the possibility exists for the fan motors toinduce motor-generated noise in the backplane power layers—noise thatcould propagate to the power circuitry for the sensitive electronics onthe router's cards. To decrease the degree to which such noise couldreach the router's cards, “isolation cutouts” are designed into thepower distribution layers.

FIG. 7A shows a section of the mask of FIG. 7. Thru-hole groups 131, 132serve fan tray power connectors in a completed router. Thru-hole groups131, 132 enjoy a fairly short and unimpeded backplane path to thebackplane power attachment points for 48VB supply and return (see FIG.3). This path does not pass near the power connectors for any routercard. But the power distributed to, in this instance, at least line cardLC13 (see FIG. 3) would tend to flow in large part past thru-hole groups131, 132 were it not for the presence of cutout 133. Cutout 133 ineffect raises the resistance of a current path between the backplanepower attachment points and the LC13 power connectors that would includethru-hole groups 131, 132, severely decreasing the share of currentcarried by such a path.

3.4.2 Copper Guard Ring on Power Distribution Layers

FIG. 7 shows a mask for the entire “panel” for a backplane, includingalignment markings, test structures (including “coupons”), and flow damstructures. After panel fabrication, the backplane board is cut from thepanel. Several issues regarding this cutting process—and the resultingboard configuration and performance—are addressed by a particular designemployed on each power plane near the board edge, as described in thissection.

The 48-volt power distribution planes are preferably patterned in amanner that leaves as much copper as possible near the board edge in thepanel, in order to decrease the possibility of board edge delaminationat these particularly thick layers. But the 48-volt power distributionplanes practically cannot extend too close to the edges of the backplaneboard for at least several reasons: tools commonly used for separatingthe backplane from the panel wear quickly when cutting through copper,and have finite tolerances; the edge of the backplane will be groundedin the final product; quite a bit of digital noise projects from theedges of the high-speed layers, which may be coupled to the distributedpower if the power planes extend to the edges of their respectivelayers; and product test laboratories such as Underwriters Laboratoriesspecify a large minimum clearance from the edge of a board to any powertrace (e.g., 62 mils minimum for one embodiment).

These seemingly conflicting design concerns are solved in the preferredembodiments using a copper guard ring 134 on power distribution layers.As shown in FIG. 7 and in the magnified section of FIG. 7B, a copperguard ring 134 is patterned around the periphery of the powerdistribution layer. Copper guard ring 134 approaches to withinapproximately 15 mils of the intended edge of the board. Holes drilledthrough the copper guard ring at regular intervals and then plated (notshown in the mask) allow the guard ring to be tied to chassis groundduring board plating. The power plane 136 is separated from copper guardring 134 by a moat 135.

The copper guard ring provides several advantages. First, because theguard ring may exist much nearer the board edge than a power trace,board edge delamination problems at power distribution layers can bereduced or eliminated. Second, in the case of inadvertent delaminationat some point on the board's periphery prior to edge plating, if theedge plating shorted, it would short harmlessly to chassis ground at theguard ring. Third, the guard ring provides an additional level ofisolation between each power plane and noise injected at the boardedges.

4 Backplane Cross-section

Although many advantages exist due to improvements in the plan layout ofthe preferred backplane embodiments, perhaps even more advantages existin the design of the backplane cross-section, i.e., how layers arearranged to work together in the material “stack” of the backplane. Twogeneral material stacks are described below. The first, a “hybrid”stack, utilizes two different types of dielectric materials in thematerial stack. The second type of stack uses a single dielectricmaterial in the material stack, but the material used is a high-speeddielectric that was previously thought to be impossible to fabricateinto a board of this thickness.

4.1 General Layer Arrangement—Hybrid Lamination Design

FIG. 8 illustrates the entire cross-section of the material stack in onepreferred backplane using a hybrid material stack. The material stack ofFIG. 8 has 34 conductive layers L01 to L34 and appropriate insulatinglayers. For each conductive layer, FIG. 8 labels that layer with a layerthickness in mils and an identifier for the layer. Layers labeled “GND”are digital ground plane layers. Layers labeled “HSn” are the high-speedsignaling layers, where n represents the layer number. Layers labeled“Signal xn” and “Signal yn” are the low-speed signaling layers. The two“A 48V” layers are the supply (“dc”) and return (“rtn”) for one powersupply, and the two “B 48V” layers are the supply and return for theother power supply. For each insulating layer, the layer is accompaniedby a description of whether the layer is a core or a b-stage layer,whether the layer is of low-speed (“LS”) material, and the finalthickness of the layer in mils.

Several general observations regarding the material stack of FIG. 8 willbe made before proceeding to a more specific description. First, thelow-speed signaling and power distribution layers use a conventionaldielectric, such as FR4 (e.g., available in the “N4000-6” product familyline from Park/Nelco), with good reflow and adhesion characteristicsthat improve the board integrity near the thicker power distributionlayers. The thinner high-speed layers use a dielectric withsignificantly lower loss at the multi-Gbps signaling rates of thepreferred embodiments, such as a thermosetting allylated polyphenyleneether (APPE, e.g., the “N6000-21” product family line available fromPark/Nelco). The dielectric material transition points occur at digitalground planes L12 and L23, which are formed on FR4 core and then bondedto N6000-21 b-stage materials.

Also notable in this material stack is that each high-speed layer (withits differential signaling traces) is formed approximately equallyspaced from and between two digital ground planes, e.g., high-speedlayer HS1 is formed on layer L03, between ground planes at L02 and L04.Similarly, low-speed signaling layers L13 and L14 are isolated from theremaining stack by two digital grounds (L12 and L15), low-speedsignaling layers L21 and L22 are isolated by two digital grounds (L20and L23), and the four power distribution layers L15 to L19 are isolatedfrom the remaining stack by two digital grounds (L15 and L20) at thecenter of the material stack. Further, the two power supply planes areplaced between the two power return planes to provide yet one more layerof isolation. The result is a material stack that efficiently manageselectromagnetic interference (EMI) to provide clean power distributionand good isolation for the high-speed signals.

One additional observation is that in order to provide thesecapabilities, the complete material stack is relatively thick comparedto prior art boards, i.e., approximately 280 mils including 34conductive layers. This required the development of new fabricationtechniques, as will be described in Section 5.

4.2 General Layer Arrangement—Single-Lamination-Material Design

A second material stack embodiment is illustrated in FIG. 9. Althoughsimilar in many ways to the material stack shown in FIG. 8, the materialstack in FIG. 9 differs in several respects. Foremost, FIG. 9 usesN6000-21 dielectric material exclusively (of several different resincontents), resulting in a different thickness (and a differentfabrication process) for some of the centermost layers, and producing athicker finished board at approximately 335 mils. Other differencesexist as well. These will be detailed during the description in Section5.

4.3 Signal Distribution

As described generally above, high-speed signals route along the tenhigh-speed signaling layers HS1 to HS10. This section describesbackplane material stack considerations for high-speed signaling.

4.3.1 High-Speed Differential Pair Cross-Section

FIG. 10 illustrates in cross-section a segment of a typical high-speedlayer 140. This segment cuts cross-wise across a differential pair 142a, 142 b and the two adjacent digital ground planes 144 a and 144 b. Twolayers of 3313 N6000-21 50.6% resin content core material 146 a, 146 bspace the bottom of pair 142 a, 142 b approximately 7.0 to 7.5 mils fromlower ground plane 144 b. Two layers of cured (after assembly) 3313N6000-21 50.6% resin content b-stage material 148 a, 148 b space the topof pair 142 a, 142 b approximately 6.0 to 6.9 mils from upper groundplane 144 a. The grain of the dielectric materials is alignedleft-to-right across the backplane.

Several trace geometries have been used in the backplane embodiments. Inone embodiment used with the hybrid material stack, traces 142 a and 142b of FIG. 10 are each 8 mils wide at the bottom, 7.6 mils wide at thetop, and 1.4 mils high (i.e., formed of 1-ounce copper). The traces areseparated by a horizontal distance (measured at their bottoms) of 16mils. In this configuration, the single-ended (even) impedance of eachconductor is approximately 45.7 ohms, whereas the differential (odd)impedance of each conductor is approximately 44.6 ohms. Thisdifferential pair configuration is thus marginally more differentialthan single-ended.

In one embodiment used with the N6000-21-only material stack, traces 142a and 142 b of FIG. 10 are each 7 mils wide at the bottom, 6.6 mils wideat the top, and 1.4 mils high. The traces are separated by a horizontaldistance (measured at their bottoms) of 17 mils. In this configuration,the single-ended impedance of each conductor is approximately 48.2 ohms,whereas the differential (odd) impedance of each conductor isapproximately 47.3 ohms. One attractive feature of this configuration isthat for N6000-21 material, the intrinsic material impedance of 48 ohmsis very close to, and between, the even and odd impedance values.

As will be described below, the differential pair impedancecharacteristics have been carefully matched to the thru-hole impedancecharacteristics to largely preserve the eye pattern for a high-speeddifferential signal passing through the backplane.

4.3.2 Nonfunctional Pads and Thru-hole Configuration for Equalization

In the preferred backplanes, the backplane utilizes a significant numberof conductive layers in order to pass a large number of signals andsupply power to circuit cards. The large number of layers results in amaterial stack—and corresponding thru-hole length—of around threehundred mils. At a high-speed signaling bitrate of 3.125 Gbps and usinga 01 bit pattern, center-to-center times between consecutive signaling“eyes” on a differential pair will be 320 ps. With a propagation speedof 6.29 mils per picosecond, the center-to-center separation betweenconsecutive eyes traveling along a differential pair is only 2000 mils,or about three times the round-trip thru-hole length. The actual eyeopening with a 01 bit pattern may be much shorter—120 ps in some cases,corresponding to a distance of about 750 mils along the pair. Thus whena thru-hole is considered for what it is electrically—a stub on atransmission line—it can be appreciated that for signaling at 3.125 Gbpsand higher rates on the preferred backplanes, thru-hole reflections canpresent a serious problem with thru-holes of such length.

It is recognized herein that it is possible to manipulate thesingle-ended impedance of the backplane thru-holes and differentialpairs to permit higher-frequency operation of a backplane. The effect ofmatching the response of the thru-holes and differential pairs can beappreciated by examining the simulated eye patterns shown in FIGS.11–14.

FIG. 11 illustrates an ideal eye pattern for a series of signaltransitions that could theoretically be launched into a backplane. The“eye” of the eye pattern is the opening between temporally adjacentsignal transitions. The differential receiver requires a minimum eye“opening” in order to detect a signal transition, i.e., the voltage onthe positive-going trace must exceed the voltage on the negative-goingtrace by at least some threshold voltage ΔV_(th) for at least someminimum amount of time ΔT_(min) before a transition can be detected. InFIG. 11, the duration of the eye opening is shown as ΔT₁. If ΔT₁ exceedsΔT_(min), a receiver should be able to distinguish the signaltransition.

FIG. 12 illustrates the type of received eye pattern that would beexpected for relatively low-speed signaling across a differential pair.Although high-frequency attenuation noticeably affects the shape of theeye, the eye is still fully open, i.e., the voltage sensed on each tracereaches, for all practical purposes, its long-term steady-state valuebetween signal transitions. The eye opening of FIG. 12 is slightlytime-shifted, but its duration ΔT₂ is not much smaller than the durationof the ideal eye opening.

FIG. 13 illustrates the type of received eye pattern that could beexpected for high-speed signaling across a differential pair withunmatched stubs at each end (similar in length to those in the describedembodiments), i.e., due to reflections at the thru-holes. High-speedsignaling stresses the eye pattern detector, as the eye cannot fullyopen between consecutive signal transitions. To compound this problem,the unmatched stubs at each end of the traces (which are coincidentallymatched to each other) can place reflections on the differentialpair—reflections that alternately constructively and destructively addto the transitioning signals. In some cases, a response similar to thatshown in FIG. 13 has been observed, where the eye begins to open andthen begins to close due to the stub reflections. In some cases, thismay cause the receiver to detect two eye openings where only one shouldexist, or to detect none at all.

FIG. 14 illustrates the type of received eye pattern that could beexpected for high-speed signaling across a differential pair withmatched stubs according to an embodiment of the invention. By control ofhow reflections occur at thru-holes, a detectable eye opening responsecan be designed without the droop shown in FIG. 13.

Therefore, in the preferred approach to designing a backplane accordingto an embodiment of the invention, the transfer functions of thethru-holes and traces are considered together in order to compensate forsignal reflections at the thru-hole stubs. Considering a transmit signalT_(x)[t] launched into one pair of thru-holes, across a differentialpair, and out a pair of thru-holes at the other end, the correspondingreceived signal R_(x)[t] can be described by the composite functionR _(x) [t]=T _(x) [t]*H _(i) [t]*L[t]*H _(o) [t],

where H_(i)[t] is the transfer function for the thru-holes that thesignal is launched in to, L[t] is the transfer function for thedifferential pair, and H_(o)[t] is the transfer function for thethru-holes that the signal is launched back out of. These transferfunctions can take into account reflection, attenuation, mode groupseparation, and other known effects in order to predict the eye patternfor a given backplane configuration.

The preferred embodiments utilize a novel approach to stub impedancecontrol in which the impedance characteristics of each thru-hole aretailored by adjusting the single-ended coupling between that thru-holeand the digital ground and/or power distribution layers through whichthe thru-hole passes. Referring to FIG. 15 a, a signaling thru-hole 170and adjacent digital ground thru-hole 180 are shown in cross-section. Inthis embodiment, thru-holes 170 and 180 have a drilled diameter of 28 to30 mils, with a hole plating of one mil minimum. The signaling thru-hole170 passes through holes in each ground and power plane, and thuscapacitively couples to each of these planes. This capacitive couplingis preferably tailored using non-functional pads (“deadpads”) on someplanes to place added capacitance at selected locations along thethru-hole.

Although other deadpad configurations can be used in an embodiment ofthe invention, the disclosed pad configurations were selected based onseveral criteria. First, any added deadpad was given the minimumdiameter that could be hit during drilling without a high probability ofthe drill bit missing the pad on one side. This allowed for the pads andtheir clearances to stay a reasonable size, and allowed more pads to beadded. Second, the pads were distributed approximately every 25% of theboard, to relieve board stress and distribute capacitance at evenintervals. Third, in the hybrid board design, each power distributionlayer received a deadpad, since those layers were thick and near thecenter of the board (the different fabrication method and longerthru-hole barrel length of the single-material embodiments allowed thepower layer deadpads to be taken out). Fourth, the clearances wereincreases on the power distribution layers since those pads were thickerand therefore had a larger area for forming a capacitor.

With the material stack shown in FIG. 15 a, a prior art signalingthru-hole without deadpads would have a capacitance of about 1.2 to 1.4pF. Signaling thru-hole 170, as shown, has a capacitance of about 1.6 to2.0 pF. When matched with the differential trace geometry shown in FIG.10, this signaling thru-hole with added capacitance (and a small amountof added inductance) can substantially eliminate high-speed signalingdroop such as shown in the eye pattern of FIG. 13.

Signaling thru-hole 170 is designed to have a specific capacitivecoupling characteristic with the ground and power distribution planes ofthe backplane. On most ground planes, such as those of layers L02 andL04, thru-hole 170 passes through a 52-mil diameter clearance (see FIG.16 for a cutaway top view of layer L02 at thru-hole 170). On the groundplanes at layers L08, L15, L20, and L27, thru-hole 170 passes through a34-mil diameter pad 172 (L08) centered in a 54-mil diameter clearance(see FIG. 17 for a cutaway top view of layer L08 at thru-hole 170,showing pad 172). On the thick power distribution planes at layers L16,L17, L18, and L19, thru-hole 170 passes through a 34-mil diameter pad174 centered in a 70-mil diameter clearance (see FIG. 18 for a cutawaytop view of layer L16 at thru-hole 170, showing pad 174). Note that onthe power distribution planes the clearances for many neighboringthru-holes merge (e.g., 170 and 182, 184, and 186), as the distancebetween the holes is less than twice the specified clearance. FIG. 19shows a cutaway view of high-speed layer HS5 (layer L11), illustratingthe functional pad 178 connected to trace 176 at that layer.

FIG. 15 b is similar to FIG. 15 a, but shows a cross-section for asingle-dielectric-material embodiment. Most notably, no deadpads areused on the power distribution layers, and the signal throughhole 171passes through a 52-mil clearance on those layers.

4.3.3 One Layer Per Switch Fabric Card

One goal of the preferred backplane designs is to design differentialsignal paths with known and controllable impedance. To this end, thehigh-speed signaling differential pairs are each designed to run betweentheir card connectors on a single plane, with no layer-swapping vias. Inorder to allow an efficient routing solution with no layer-swapping,nine of the high-speed layers are each dedicated to signaling to andfrom a single switch fabric card. For example, FIG. 20 illustrates thepanel mask for layer L07 (high-speed signaling layer HS3), whichconnects switch fabric card SFI (FIG. 3) to each of the line cards. Itcan be appreciated that the resulting layout allows for differentialpair routing that is largely direct and short. The tenth high-speedlayer is used for short-reach signaling to several switch fabric cards,as will be explained next.

4.3.4 Selection of a High-Speed Layer for Signal Routing

Not only have the differential pairs been arranged for efficientrouting, but the selection of which high-speed layer(s) will be used foreach switch fabric card also improves performance. Generally, the lowerhigh-speed layers have been designed to carry the very longesthigh-speed traces, and the longest traces have been avoided on the upperhigh-speed layers.

Referring to FIG. 21, a simplified cross-section of a backplane 150 isillustrated. Cross-section 150 shows a first trace 154 and a secondtrace 160. First trace 154 is located on a lower high-speed layer, andconnects to two thru-holes 152 and 156. Second trace 160 is located onan upper high-speed layer, and connects to two thru-holes 158 and 162. Asignal launched into thru-hole 152 travels down trace 154, but alsotravels down the remainder of thru-hole 152, which forms a stub 164 thatreflects the signal back in the other direction. Likewise, a signallaunched into thru-hole 158 travels down trace 160, but also travelsdown the remainder of thru-hole 158, which forms a much longer stub 166that reflects the signal back in the other direction. Similarreflections occur at the exit ends of traces 154 and 160.

Longer stub 166 produces a much more problematic reflection than shorterstub 164. One way that this tendency is compensated for in the preferredembodiments is by routing shorter traces on the upper high-speed layers(where the reflections are more significant but the signal is not asdegraded due to a long propagation path) and routing longer traces onthe lower high-speed layers (where the reflections are less significantand thus longer path lengths, with more attenuation, can be tolerated).

In conjunction with the goal of dividing traces by dedicating high-speedlayers to switch fabric cards, the preferred embodiments use at leasttwo techniques to select routing layers. First, the topmost layer—withthe longest stubs—is not dedicated to a single switch fabric card, asthis would require some longer traces to reach the outboard line cards.Instead, HS1 serves a group of connections that are fairly short becausethese line cards are substantially vertically aligned with thecorresponding switch fabric cards. Some connections meeting thiscriteria, and thus selected for HS1, are: line cards LC11, LC12, andLC13 to switch fabric card SF8; line cards LC9 and LC10 to switch fabriccard SF7; and line cards LC7 and LC8 to switch fabric card SF6.

A second technique for reducing trace length on the upper layers is toselect line card connector pins in a manner that results in shorterlengths for the upper layers. For instance, looking at FIGS. 3 and 20 inconjunction, it can be appreciated that the traces connecting SF1 to theline cards do not use the upper set of connector blocks (JLC4A and itscounterparts for the other cards), but use the lower pins of JLC4C andthe pins of JLC4B, and their counterparts. This reduces the maximumtrace length on layers HS2, HS3, and HS4 by several inches. Startingwith layer HS5, pins in JLC4A and its counterparts are used, startingfrom the bottom. FIG. 4 shows typical pinouts for HS9. Thus although allhigh-speed layers contain some short traces, those with the very longesttraces are those with the shortest via stubs. Looked at strictly from atrace length standpoint this is counterintuitive, since inserting thelongest horizontal trace signals to and extracting them from thebottommost high-speed layer adds even more length—almost two-thirds ofan inch in the described embodiments—to those signal paths as comparedto a comparable path on the topmost layer.

4.3.5 Low-Speed Signal Distribution

Some backplane signaling does not operate at high switching speeds. Thepreferred embodiments designate a number of layers for use with suchsignals. Typically, these signals are single-ended signals for partybuses, clock distribution, etc. In FIG. 8, layers L13 (Signal x1), L14(Signal y1), L21 (Signal y2), and L22 (Signal x2) are used for low-speedsignaling. Note that L13 and L14, as well as L21 and L22, do not have aground plane interposed between them. These low-speed signaling layerpairs are, however, separated from adjacent high-speed layers by adigital ground plane. The low-speed signaling layer pairs are alsoseparated from the power distribution layer pairs by a digital groundplane. This arrangement, which pushes the higher-speed signaling andreturn ground noise-with accompanying higher levels of EMI—further fromthe power distribution layers, serves to further isolate powerdistribution from EMI.

4.4 Power Distribution and Noise Isolation

Although several aspects of the matter have been discussed previously,the preferred embodiments use layer stacks that allow economicaldistribution of power with superior noise isolation. The preferredarrangements of layers and choice of materials allow a significantamount of noise-isolated power (some embodiments are rated at 200amperes for each power supply) to be distributed within a common boardthat also serves the high-speed and low-speed signaling needs of therouter.

4.4.1 Arrangement of Embedded Power Layers

The particular designs of the preferred material stacks have severaladvantages over previous designs. As just discussed, the buried andground-plane-isolated power supply planes L16–L19 provide a relativelynoise-free power distribution system for the router. Were the powerdistributed by conventional means such as bus bars, roughly twice asmuch power conditioning and filtering would be required on each card toachieve similar noise characteristics. By placing the power supplyplanes buried between two isolating ground planes L15 and L20 andkeeping power supply connections relatively isolated from signalingconnections, the designs shown in FIGS. 8 and 9 avoid the need forcostly power distribution components.

Placing the power supply planes in such a thick material stack causesother difficulties, however. In order to keep resistance low, the powersupply planes should be relatively thick, e.g., three- or four-ouncecopper. The preferred dielectric materials for the high-speed layers donot fill gaps between relatively thick traces well, and therefore havebeen adapted herein for use in some embodiments using special processes.Furthermore, the preferred high-speed dielectric materials are generallyill-suited for use in such a thick material stack, as the stressconcentrated at the locations of the thru-holes tends to cause splittingand cracking during thru-hole drilling. Two preferred approaches havebeen developed for dealing with these problems while allowing use ofN6000 or similar dielectric material on the high-speed signaling planes.

4.4.2 Hybrid Lamination Design

In the approach shown in FIG. 8, the use of low-speed FR4 material onall layers between L12 and L23 allows the thick material stack withburied power distribution planes to be fabricated without creatingvoids. In one preferred embodiment, two 1080 N4000-6 glass sheets with aresin content of 57.5% are used on each side of each low-speed signalinglayer; for the power distribution layers, 1080 N4000-6 sheets with ahigher resin content (63.5%) are used to enhance gap filling. Two coresheets are used between each power distribution supply and return planepair. Three b-stage sheets are used between each power distributionreturn plane and the neighboring digital ground plane. And four b-stagesheets are used between the adjacent supply planes. In this embodiment,the interface between N6000 and FR4 occurs at ground planes (L12 andL23), such that copper largely separates the two dielectric materialsand good bonding is achieved.

The via-capacitance-tailoring pads used on layers L15 through L20 servea second purpose in that they aid the manufacturability of the board.Stress at the high-speed connector vias, as well as the size of thevoids that must be filled during booking, are lessened by the use ofdeadpads on the power distribution planes and adjacent digital groundplanes.

This material stack has an added advantage for noise rejection. Notethat because FR4 has higher losses for high frequency signals thanN6000, the use of FR4 near the power distribution planes provides anadditional measure of high frequency noise rejection between the powersupply and low-speed signaling layers.

4.4.3 Pure N6000 Lamination Design

A second fabrication approach can produce a material stack such as shownin FIG. 9. This material stack preferably uses N6000-21 materialexclusively, for high-speed, low-speed, and power distribution. Theresin contents are modified near the center of the board to enhancevoid-filling as follows: two sheets of 3313, 50.6% resin contentN6000-21 glass form all cores in the board except for the cores betweenL16 and L17, and between L18 and L19, where two sheets of 1080, 60.3%resin content N6000-21 glass are used. Each of the two patterned powercores is sandwiched between two b-stage sheets of 3313, 53.4% resincontent and one b-stage sheet of 1080, 65% resin content N6000-21 glassper side in a first booking designed to fill the deep voids in thepatterned four-ounce copper planes. An additional four sheets of 1080,60.3% resin content b-stage material are placed between layers L17 andL18 prior to final booking, with three such layers placed between L15and L16, and between L19 and L20. Two sheets of b-stage 1080, 60.3%resin content are also placed between L01 and L02, between L13 and L14,between L21 and L22, and between L33 and L34. Like the hybrid design,b-stage material for the high-speed layers consists of two 50.6% resincontent 3313 N6000-21 glass sheets.

An advantage of the pure N6000 design over the hybrid design is that itresolves any dielectric-compatibility issue that may exist with thehybrid design.

5 Fabrication of a High-speed, High Layer Count Backplane

Preferred methods for assembling the layers of a backplane will now bedescribed. A process for single-material, multiple-lamination-cyclefabrication will be described first in its entirety. A process forsingle-lamination-cycle, hybrid-material fabrication then will bedescribed where it differs from the first process.

5.1 Single Material/Multiple Lamination Cycle Design

One preferred method of making a backplane embodiment uses a singlehigh-speed dielectric material throughout. To improve themanufacturability of such a design, multiple lamination cycles are usedto complete a panel.

5.1.1 Core Makeup

Prior to assembly of the backplane, a first step in the fabrication ofthe backplane is the makeup of plated and patterned core sheets. For ahigh-speed layer, a preferred core sheet consists of two sheets of 50.6%resin content 3313 N6000-21, which are laminated together underlaminating conditions as recommended by the manufacturer to cure them.Once bonded and cured, these two sheets form a core dielectric layerabout 7.5 mils thick, with peak roughness features on the order of 0.1mils. Although from a mode group separation viewpoint this roughnesswould not necessarily be desirable, the desire for a smooth conductorsurface is preferably balanced by the need for good adhesion betweenN6000 and copper.

The core is plated with one-ounce copper on both sides. The copper onone side is patterned using an etch-compensated process to produce oneof the desired layers of high-speed differential pairs; the copper onthe other side is patterned using a similar process to produce theadjacent ground plane. In one preferred embodiment, tendifferently-patterned copper-plated cores form the ten high-speed layersthat will be assembled in the finished product, and another fourdifferently-patterned cores form the four low-speed layers that will beassembled in the finished product.

After patterning, the patterned cores are processed through an oxidetreatment process that roughens the outer surfaces of the copperplating, as well as cleans them, to enhance copper-to-b-stage adhesionduring the lamination cycles. Preferably, the parameters of this processare controlled to produce a copper surface roughness similar to thatfound at the plating-to-core-dielectric boundary. It is believed thatadjusting the top-surface and bottom-surface trace roughness to beapproximately equal prevents additional mode group separation, as thecurrent traveling along the top and bottom of the traces will incursimilar delays due to surface roughness.

The two power cores are prepared in somewhat similar fashion. Two sheetsof 60.3% resin content 1080 N6000-21 material are laminated togetherunder laminating conditions as recommended by the manufacturer to curethem. Once bonded and cured, these two sheets form a core dielectriclayer about 6 mils thick, with peak roughness features on the order of0.5 mils. As the power layers do not pass high frequencies, the largersurface roughness is preferred in order to increase metal-to-dielectricadhesion.

Each power core is plated with four-ounce copper on both sides. Thecopper on one side is patterned using an etch-compensated process toproduce one of the desired DC supply planes; the copper on the otherside is patterned using a similar process to produce the correspondingDC return plane. In the preferred embodiments, two differently-patternedcopper-plated cores form the two sets of power planes used in thebackplane.

After patterning, the patterned power cores are processed through anoxide treatment process that roughens the outer surfaces of the copperplating to enhance copper-to-b-stage adhesion during the laminationcycles. Preferably, the parameters of this process are controlled toproduce a copper surface roughness similar to that found at theplating-to-core-dielectric boundary, i.e., 0.4 to 0.5 mils for the powerplanes.

5.1.2 First Lamination Cycle

In the preferred dual-lamination-cycle embodiments, a first laminationcycle bonds two sheets of 3313 53.4% resin content and one sheet of 108065% resin content N6000-21 material, to each side of the two power cores(layers L16/17 and L18/19, respectively). This separate laminationcycle, performed with high-resin-content glass, ensures that thefeatures in the four-ounce patterned power planes are filled with glassand void-free. The first lamination cycle is performed under laminatingconditions as recommended by the manufacturer.

The 1080 material is placed on the outside layers of the subassembly.The 3313 material is rich in resin and freely gives up that resin to bepressed into etched copper areas. Unfortunately, this can leave areas onthe subassembly surface without enough pressure during lamination toadhere the glass to the core. The 1080 material fills in, thus avoidingthese areas of low pressure and producing a smooth, fully laminatedsheet in preparation for the final lamination cycle.

5.1.3 Providing for Alignment

Panel masks conventionally contain alignment marks (see marks 202, 204in FIGS. 7 and 20). These marks are registered in each layer such thatwhen the material stack is built up prior to booking, the layers can beco-aligned by aligning the marks.

The dual lamination cycle presents a problem with respect to alignment.Once a conductive layer, e.g., L16, has been laminated to glass duringthe first lamination cycle, the alignment marks are obscured and cannotbe used to align the layers in the second lamination cycle with thedesired accuracy.

In the preferred embodiments, notching the glass sheets prior to thefirst lamination cycle solves this problem. As illustrated in FIG. 22 a,a power core 210 has alignment marks 202 and 204. Six glass sheets 220,221, 224, 225 (3313 material) and 222, 226 (1080 material) are cut tosize, and then notched at locations 206, 208 where the glass sheets willoverlay the alignment marks. After the first lamination cycle, glasssheets 220, 222, 224 and 226 are bonded to power core 210, but do notobscure alignment marks 202 and 204 (see FIG. 22 b) in the intermediateassembly 230. Consequently, the second lamination cycle can rely onmarks 202, 204 to properly align the intermediate assemblies with theother cores.

It is acknowledged that in the final panel assembly, either very poorfill—or no fill—may be observed at the location of notches 206, 208. Thealignment marks are placed in noncritical locations, far from the boarditself or any coupons such that delamination near the alignment marks isof little concern.

5.1.4 Second Lamination Cycle

The backplane panel is formed by stacking and aligning thecopper-patterned cores from the different high-speed and low-speedlayers with intermediate power core assemblies from the first laminationcycle, in the order depicted in FIG. 9. As shown in FIG. 9, wherehigh-speed layers are adjacent, the ground plane of one high-speed layerfaces the high-speed traces of the adjacent high-speed layer, with twosheets of 50.6% 3313 N6000-21 b-stage glass interposed. Adjacentlow-speed layers are also stacked with two interposed b-stage sheets,but the material has 60.3% resin content. Between ground plane L15 andthe intermediate glass layers laminated to power return L16, threesheets of 60.3% 1080 N6000-21 b-stage glass are interposed. The samearrangement is interposed between layers L19 and L20. Four such glasssheets are interposed between the two intermediate power core assemblies(between layers L17 and L18).

Once the copper-patterned cores and the b-stage sheets are stacked andaligned, the material stack is placed in a booking press. The entirestack is booked under laminating conditions as recommended by themanufacturer.

After the material stack is cooled, the thru-holes are drilled in thebackplane, and the entire assembly is plated with one-ounce copper. Thepads are then patterned, and a protective mask is added to complete theboard. Connectors are then press-fit to the appropriate locations of theboard to complete the backplane assembly.

5.1.5 Drill Cycle

Due to the thickness of the material stack and the multiple metal padsthat are drilled through during via fabrication, the drill bit canbecome hot. This raises the probability that the glass may become hotenough to allow the bit to “spin” a deadpad or a conductive pad.Spinning a pad refers to the pad delaminating or tearing loose from theglass under drilling pressure, thereby damaging the board and possiblyruining it.

To avoid spun pads, the preferred drill cycle, and the board itself,have been designed to keep the drill bit cool. The drill is programmedto perform a “multi-peck” drill cycle for each critical via. A firstpeck of the drill bit penetrates the board to approximately halfwaybetween layers L17 and L18, i.e., halfway through the board. As thenominal thickness of the dielectric between L17 and L18 is 24 mils,there is considerable margin for error in setting the depth of thispeck. It is, however, believed to be important that the first peck notend right at a conductive layer, as this may cause the drill bit to graband tear the copper pad when inserted for the second peck.

After the first peck, the drill bit is extracted from the partial viabriefly, allowing heat to dissipate from the bit and the partial via.The drill bit then performs a second peck at the same location, thistime penetrating through to the bottom of the board.

If the board-thickness variance from board to board or lot to lot issignificant, it may not be sufficient to program the drill with a presetdepth for the first peck. In such case, an unneeded portion of the panelcan be sectioned and measured in order to adjust the drill depthindividually for each board or lot.

5.1.6 Thieving

Very little of the plated copper on each high-speed and low-speed layeris actually needed to form the signaling traces (see, e.g., FIG. 20). Onthe other hand, each of these layers shares a core with a ground planelayer (see, e.g., FIG. 24) that uses a great deal of the copperoriginally plated on the core. Because of this disparity in coppercoverage, it has been found that the patterned cores tend to curl,making them difficult to work with. Further, it has been found thatduring the booking process, the high-speed and low-speed traces tendedto migrate slightly towards the edges of the board, resulting inmisalignment in the final panel. Thieving also helps in maintaining aconsistent dielectric thickness across the board, which provides abenefit of better impedance uniformity.

To combat these problems, the preferred embodiments use “thieving” inthe signaling layer masks. In the present disclosure, thieving consistsof a pattern, such as pattern 240 in FIG. 20, of unconnected coppermesas in areas of the board that are trace-free and via-free. Becausethe preferred embodiments avoid layer-swapping vias, the non-connectorregions of the board are generally via-free and suitable for thieving.

FIG. 23 illustrates a magnified section of panel 200 from FIG. 20 inorder to better illustrate the use of thieving. The thieving pattern 240on the high-speed layers is laid out in a grid pattern. Each “dot” is a50-mil diameter copper mesa. The dots are spaced 75 milscenter-to-center in a grid pattern. On the high-speed layers, a 150-milspacing is maintained between the thieving pattern and the closest traceor via. On the low-speed layers, a 100-mil spacing is maintained betweenthe thieving pattern and the closest trace or via, on either the samelayer or the neighboring low-speed layer. Also, for neighboringlow-speed layers (the L13/L14 pair, and the L21/22 pair), the thievingpattern is interlaced in a “star-dot” pattern as shown in FIG. 25.

5.1.7 Flow Dams

On each layer, the panel outside of the board region is designed withflow dams 250 (see FIG. 20). The flow dams resist the flow of resin outthe sides of the panel during booking, thereby forcing the resin to fillinternal voids in the board pattern as much as possible. But the flowdams do not completely stop the flow of resin—a controlled flow keeps asmuch resin as possible without creating pressure at the position wherethe outer edge of the board will be routed from the panel. It isbelieved that this step reduces the chance of delamination at the outeredge of the board during routing.

5.2 Hybrid-Lamination Design

The preferred core makeups for the high-speed layers in the hybridlamination design are identical to the core makeup described for thedual lamination cycle design. The low-speed cores and power cores aredifferent, however. Each low-speed or power core is made of 1080N4000-6, with a 57.5% (low-speed) or 63.5% (power) resin content. Thepower cores are made of two glass sheets, and the low-speed cores aremade of two glass sheets.

It has been found that dual lamination cycles can be avoided withN4000-6 as that material flows and reflows much easier than N6000-21.Thus once the cores have been roughened as described for the duallamination cycle design, all cores are stacked for booking. B-stageglass sheets for the high-speed cores are identical to the sheetsdescribed for the dual lamination cycle. Two glass sheets of 57.5% 1080N4000-6 are placed between adjacent low-speed cores. Three glass sheetsof 63.5% 1080 N4000-6 are placed between the adjacent low-speed andpower cores. And four sheets of 63.5% 1080 N4000-6 are placed betweenthe two power cores.

5.2.1 Lamination Cycle

A single lamination cycle is used to book the hybrid panel. Laminatingconditions as recommended by the manufacturer are used.

5.2.2 Teardrop Pad Construction

Like with the dual lamination cycle design, precautions have been takenwith the hybrid design to greatly reduce the possibility of spun pads.It is believed that FR4 reflows much easier than N6000 due to the heatof drilling and this increases the propensity for spun pads in the FR4portion of the material stack. Like with the dual lamination cycledesign, a multi-peck drill cycle is used. But in addition, signal padsin the low-speed layers are formed as teardrop pads, i.e., eachlow-speed pad is augmented with a “half pad” displaced from the firstpad to form a teardrop pad. The teardrop is directed towards the traceconnected to the pad. FIG. 24 illustrates the construction of oneteardrop pad 260.

Teardrop or oversize pads are sometimes used with lower-cost boards tocompensate for poor drill tolerances. But the inventors believe this tobe the first use of such pads in a design that does not need teardroppads to compensate for a poor drill process, where during drilling thetolerance allows the via to be offset towards the end of the teardrop.This sturdier pad is simply much better at resisting spinning duringdrilling.

5.2.3 Drill Cycle

A multi-peck drill cycle using three separate pecks is preferred for thehybrid design. Three pecks allows the drill to cool once before enteringthe FR4 portion of the board, and once just after leaving the FR4portion of the board.

The preferred endpoint for the first peck is in the dielectric layerbetween HS4 (layer L09) and the underlying digital ground plane at layerL10 (see FIG. 8). This endpoint is selected to ensure that any pad atlayer L09 will have already been drilled through, and thus will not bespun on the second peck. Also, when the drill is inserted for the secondpeck, at L10 it will encounter either solid copper or an empty viaspace, but not a pad.

Similar considerations lead to a preferred endpoint for the second peckin the dielectric layer between Signal x2 (layer L22) and the underlyingdigital ground plane at layer L23.

One of ordinary skill in the art will recognize that the concepts taughtherein can be tailored to a particular application in many otheradvantageous ways. Although specific high-speed and low-speed dielectricmaterials are used in the preferred embodiments, the principle of usinga different dielectric material or different resin content for thicker,embedded power distribution planes can be adapted to other materials andmaterial stacks. The material stack need not be symmetric about itscenter as shown in the preferred embodiments. As another example, theprinciple of routing differential pairs past ground thru-holes bysplitting them around those holes can be adapted to connector patternsother than those used by the AMP HS3 connector geometry. Although abackplane embodiment has been disclosed, the concepts taught hereinapply equally to other interconnection arrangements such as midplanes.And in other designs, the concept of allocating signal planes toindividual switch fabric cards could be reversed, allocating signalplanes to individual line cards.

Although the specification may refer to “an”, “one”, “another”, or“some” embodiment(s) in several locations, this does not necessarilymean that each such reference is to the same embodiment(s), or that thefeature only applies to a single embodiment.

1. An electrical router backplane comprising: a first plurality of cardslots, each configured for mating with the backplane electricalconnector sets of a packet input/output card to couple electricalsignals from that packet input/output card to a corresponding set ofsignaling thru-holes in the backplane; a second plurality of card slotscomprising at least three card slots, each configured for mating withthe backplane electrical connector sets of a switch fabric card tocouple electrical signals from that switch fabric card to acorresponding set of signaling thru-holes in the backplane; and multipleembedded high-speed signaling layers, each sandwiched betweencorresponding upper and lower dielectric material layers separating thatsignaling layer respectively from upper and lower ground planes, eachhigh-speed signaling layer comprising differential trace pairs, eachtrace pair forming point-to-point connections between two pairs of thesignaling thru-holes, such that, collectively, the high-speed signalinglayers contain at least two differential trace pairs to connect eachfirst card slot with each second card slot.
 2. The backplane of claim 1,each differential trace pair configured with an individual trace widthof approximately 7 to 9 mils and a trace height, differential tracespacing, and trace-to-ground-plane spacing related to trace width suchthat the impedance of a given trace is marginally more differential thansingle-ended.
 3. The router backplane of claim 2, wherein the individualtrace width is approximately 8 mils, the differential trace height isapproximately 1.4 mils, the differential trace spacing is approximately16 mils, and the trace-to-ground-plane spacing is approximately 6 to 8mils.
 4. The router backplane of claim 2, wherein the individual tracewidth is approximately 7 mils, the differential trace height isapproximately 1.4 mils, the differential trace spacing is approximately17 mils, and the trace-to-ground-plane spacing is approximately 6 to 8mils.
 5. The router backplane of claim 2, wherein the differential andsingle-ended impedance values of a given trace are approximately equalto the intrinsic impedance of the dielectric material.
 6. The routerbackplane of claim 5, wherein the differential impedance is slightlyless than the intrinsic impedance and the single-ended impedance isslightly greater than the intrinsic impedance.
 7. The router backplaneof claim 2, each card slot having a connector region comprising rows ofsignaling thru-holes alternating with rows of ground thru-holes, theground thru-holes connected to the ground planes, wherein eachhigh-speed signaling layer has a plurality of differential trace pairsrouted through one or more connector regions such that, for any givenone of those trace pairs, that trace pair passes through a connectorregion in an alignment that intersects a row of ground thru-holes, suchthat where that trace pair approaches a ground thru-hole, the traces ofthat differential trace pair separate as they approach that groundthru-hole, pass on opposite sides of that thru-hole, and rejoin into adifferential configuration on the opposite side of that thru-hole. 8.The router backplane of claim 7, wherein as the traces of a differentialpair pass on opposite sides of a ground thru-hole, each of those twotraces maintains a separation from that thru-hole approximately equal tothe trace-to-ground-plane spacing for that differential pair.
 9. Therouter backplane of claim 7, wherein the alignment of a given one ofthose differential trace pairs routed through a connector region iscentered with respect to the thru-hole, and wherein the two traces ofthat pair separate, as they approach the thru-hole, at approximately a90-degree angle until the two traces are separated by more than theclearance required for the thru-hole, pass the thru-hole parallel toeach other, and then rejoin at approximately a 90-degree angle untilreaching the differential configuration.
 10. The router backplane ofclaim 1, wherein the initial segments of two traces of a differentialpair are length-matched by the inclusion of a looped jog near thesignal-insertion end of the nominally shorter of the two trace segments.11. The router backplane of claim 10, wherein the jog furthercompensates for path length differences in a card backplane electricalconnector, between the connector signal paths corresponding to the firstand second traces.
 12. The router backplane of claim 1, wherein thefirst and second pluralities of card slots are arranged respectively infirst and second ranks, the first and second ranks spaced apart to forma differential trace routing area between the ranks.
 13. The routerbackplane of claim 12, wherein, for the majority of the high-speedsignaling layers, each layer contains substantially all of thedifferential trace pairs connecting each first card slot to a selectedone of the second card slots.
 14. The router backplane of claim 12,further comprising power distribution planes embedded between adjacentground plane layers, each card slot coupled to the power distributionplanes at power thru-holes corresponding to power connectors on thecards, the power thru-holes located outboard of the signaling thru-holesfor each card slot.
 15. The router backplane of claim 12, furthercomprising a card slot for a route processing module, that card slothaving substantially more signaling thru-holes than either a packetinput/output card slot or a switch fabric card slot, the routeprocessing module card slot located within the first rank of card slots,substantially in the middle of that rank.
 16. The router backplane ofclaim 1, wherein the longest differential pairs are routed on thebottommost high-speed signaling layers, with the backplane bottomhigh-speed signaling layer defined as the layer furthest from the cardslot side of the thru-holes connected to those differential pairs. 17.The router backplane of claim 1, wherein the high-speed signaling layersare arranged within the backplane according to the longest differentialpairs on each signaling layer, such that the topmost signaling layershave shorter longest differential pairs than the bottommost signalinglayers, with the backplane bottom high-speed signaling layer defined asthe layer furthest from the card slot side of the thru-holes connectedto the differential pairs.
 18. The router backplane of claim 1, furthercomprising low-speed signaling traces, the correspondence of signalingthru-holes to traces arranged, for the majority of the high-speeddifferential pairs, such that high-speed differential pairs connect tosignaling thru-holes adjacent to signaling thru-holes that are eitherconnected to low-speed signaling traces or unconnected.
 19. The routerbackplane of claim 18, wherein the signaling thru-holes are arranged inrows of six holes, such that for the majority of the rows of signalingthru-holes, the outermost two holes on each end of a row connect to thetraces of a high-speed differential pair, and the innermost twothru-holes do not.
 20. The router backplane of claim 18, wherein themajority of the low-speed signaling traces reside on low-speed signalinglayers separate from the high-speed signaling layers.